{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "24d78855-9712-419b-8201-486452f5120a",
   "metadata": {},
   "source": [
    "# Pandas Period"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdb234fa-ee2e-403d-a0ff-4b2c0fdced43",
   "metadata": {},
   "source": [
    "[Feature Engineering for Time Series Forecasting](https://www.trainindata.com/p/feature-engineering-for-forecasting)\n",
    "\n",
    "In this notebook we'll discuss the Pandas `Period` and `PeriodIndex` type to handle time span related data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "010aee50-728d-4c24-a6f6-9282a71364c1",
   "metadata": {},
   "source": [
    "# Load example data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55065bd0-a3fe-4d4b-970c-ea09d514fb12",
   "metadata": {},
   "source": [
    "The air passengers dataset is the monthly totals of international airline passengers, from 1949 to 1960, in units of 1000s. \n",
    "\n",
    "For instructions on how to download, prepare, and store the dataset, refer to notebook number 5, in the folder \"01-Create-Datasets\" from this repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7e30d3c0-baa1-4fb0-86c4-6196e46641c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d8d74785-9082-4711-8dad-de0d3b333ab6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\n",
    "    \"../Datasets/example_air_passengers.csv\",\n",
    "    parse_dates=[\"ds\"],\n",
    "    index_col=[\"ds\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f258096a-1171-43b0-97d7-70ce59f74e00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['1949-01-01', '1949-02-01', '1949-03-01', '1949-04-01',\n",
       "               '1949-05-01', '1949-06-01', '1949-07-01', '1949-08-01',\n",
       "               '1949-09-01', '1949-10-01',\n",
       "               ...\n",
       "               '1960-03-01', '1960-04-01', '1960-05-01', '1960-06-01',\n",
       "               '1960-07-01', '1960-08-01', '1960-09-01', '1960-10-01',\n",
       "               '1960-11-01', '1960-12-01'],\n",
       "              dtype='datetime64[ns]', name='ds', length=144, freq=None)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "777e7fb4-3554-41e4-8a0d-8b1d8086e14f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas._libs.tslibs.timestamps.Timestamp"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(df.index[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9d5b80c-3b03-427e-a474-de54022904b9",
   "metadata": {},
   "source": [
    "The current type of our index is a `DatetimeIndex` where each element is a `Timestamp`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf881c8f-77b0-4dea-89a3-9a39046a5e64",
   "metadata": {},
   "source": [
    "# Pandas Period - what is it and when to use it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e98b0a0-27a6-4515-91fa-961a76ae0a8a",
   "metadata": {},
   "source": [
    "When working with time related information which refers to a time span (e.g., the sales of products over each month) rather than an instance in time (e.g., an event that occurs at a specific timestamp), it can be more convenient to work with a data type in Pandas called `Period`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "995f40b2-fab4-40a0-9b43-643da70f2b56",
   "metadata": {},
   "source": [
    "To read more about the `Period` type in Pandas see the [docs](https://pandas.pydata.org/docs/user_guide/timeseries.html), in particular the section titled \"timestamps vs. time spans\".\n",
    "    \n",
    "   > \"A `Period` represents a span of time (e.g., a day, a month, a quarter, etc).\"\n",
    "   \n",
    "   > \"Under the hood, pandas represents timestamps using instances of `Timestamp` and sequences of timestamps using instances of `DatetimeIndex`. For regular time spans, pandas uses `Period` objects for scalar values and `PeriodIndex` for sequences of spans.\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d47571a-47c0-4611-b5a1-f90e456a72eb",
   "metadata": {},
   "source": [
    "`Period` objects can be created just as easily as timestamp `Timestamp` objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f979b672-257f-459e-9b55-84d4aab3760b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-01 00:00:00')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp(\"2020-01-01\") # Create a timestamp representing 1st January 2020 at time 00:00:00"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1ee94d2b-94a0-4f7e-9a86-5ee68430f8b2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Period('2020-01', 'M')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Period(\"2020-01\", freq=\"M\") # Create a time period representing the month of January 2020"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb8d418b-64b5-4c1b-a214-fe64a2e7eb5d",
   "metadata": {},
   "source": [
    "For example, our dataset index currently is a `DatetimeIndex` where there is a day (and even a time) associated with each month (e.g., 1960-12-01 00:00:00), despite the day and time being meaningless for this data set. What we're trying to represent is the sales over the time span of a given month."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "463cb570-fbab-4a1b-926e-ff911d628868",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ds</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1949-01-01</th>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-02-01</th>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-03-01</th>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-04-01</th>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-05-01</th>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              y\n",
       "ds             \n",
       "1949-01-01  112\n",
       "1949-02-01  118\n",
       "1949-03-01  132\n",
       "1949-04-01  129\n",
       "1949-05-01  121"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c107454-d49a-49e4-adbf-2c0a97f6597d",
   "metadata": {},
   "source": [
    "We can convert the index from `datetime` to `Period` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bba662e9-82ee-47a3-ad78-88047a12a911",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.index = df.index.to_period()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "eda8d569-a678-4d35-90a6-d288cf53986d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ds</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1949-01</th>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-02</th>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-03</th>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-04</th>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1949-05</th>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           y\n",
       "ds          \n",
       "1949-01  112\n",
       "1949-02  118\n",
       "1949-03  132\n",
       "1949-04  129\n",
       "1949-05  121"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e6d889f0-8c74-4eec-a367-da6c005b067c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PeriodIndex(['1949-01', '1949-02', '1949-03', '1949-04', '1949-05', '1949-06',\n",
       "             '1949-07', '1949-08', '1949-09', '1949-10',\n",
       "             ...\n",
       "             '1960-03', '1960-04', '1960-05', '1960-06', '1960-07', '1960-08',\n",
       "             '1960-09', '1960-10', '1960-11', '1960-12'],\n",
       "            dtype='period[M]', name='ds', length=144)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f351009-6800-470c-845c-5f7338a8db97",
   "metadata": {},
   "source": [
    "We now have a `PeriodIndex` with monthly frequency which better represents the time series (i.e., the sales over the whole month)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e2f4e38-9bee-46dc-88b1-8933bf5a0394",
   "metadata": {},
   "source": [
    "`Period` objects can make it easier to do certain calculations. Let's add one month to a given period:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "9eae6521-ed45-4bb9-b1c4-c375c5ebdf6b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Period('1949-01', 'M')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e86cbaa9-ce01-47cc-b5e0-b8ef6931d77e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Period('1949-02', 'M')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index[0] + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d567f8f-a0f7-4c3d-84ba-4385f44daeb5",
   "metadata": {},
   "source": [
    "`Period` is also the preferred type when calculating the **exact** differences in dates in terms of calendar events (e.g., what is the exact integer difference between the week numbers of the two following timestamps: \"2012-01-15 10:00:00\" (week 2, year 2012) and \"2014-04-01 01:30:00\" (week 14, year 2014))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95c877ef-d1a1-4f44-a0e1-5f7c700cc064",
   "metadata": {},
   "source": [
    "Using `Period`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "fd40ddc1-00bd-4289-a018-27a918d72e68",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<-116 * Weeks: weekday=6>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "delta = pd.Period(\"2012-01-15 10:00:00\", freq=\"W\") - pd.Period(\"2014-04-01 01:30:00\", freq=\"W\")\n",
    "delta"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f43b42-ffcc-4f7e-ad2e-133bb2a056c8",
   "metadata": {},
   "source": [
    "We can get the integer using the `n` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "bfa9b0fb-3e19-4465-baaa-d99d0cebb9d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-116"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "delta.n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "050c54fa-fc33-4b6e-97bb-e594fe148897",
   "metadata": {},
   "source": [
    "Using `Timestamp` and `timedelta` objects we only get approximate, and sometimes incorrect, answers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "be49da06-d33f-4be8-9a6b-443450331a92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-115.23511904761905"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(pd.Timestamp(\"2012-01-15 10:00:00\") - pd.Timestamp(\"2014-04-01 01:30:00\")) / np.timedelta64(1, \"W\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3119df1a-2812-4133-b9e6-4e7580c8cf64",
   "metadata": {},
   "source": [
    "Whether we use `Period` or `datetime` should not change the forecasting workflow, but it will make some calculations easier depending on the time series."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3bf4812-70e1-4a36-9658-90d164395bc8",
   "metadata": {},
   "source": [
    "In general, if your data represents a timespan then `Period` (e.g., sales over one month) can make handling the data more convenient. If your data represents events that occurred at a timepoint then `datetime` or `Timestamp` is preferred."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
